09/762837 

12 Rec'd PCT/PTO 1 3 FEB 2001 



Description 

System for enabling self -monitoring, with regard to 
body movement sequences to be carried out, by the 
moving person 

The invention relates to a system for enabling 
self -monitoring, with regard to body movement sequences 
to be carried out, by the moving person. 

Practising specific movements or movement 
sequences plays an important part for example in the 
context of rehabilitation. In this case, the subject or 
patient practises specific movement sequences in order 
to train his/her overall body mobility or, 
alternatively, in order, for example , to influence 
specific body parts or muscle groups in a targeted 
manner. However, the targeted practising of specific 
movement sequences is also an important therapeutic 
element for physically disabled persons. When 
practising these movement sequences or when carrying 
out the training exercises, it is often crucial that 
the movements be performed "correctly", that is to say 
that a predetermined movement sequence be adhered to in 
the best possible manner. It would be desirable here to 
identify deviations as far as possible in the course of 
the movement, in order to be able to correct them 
immediately, thereby avoiding the situation where an 
exercise is repeatedly carried out "incorrectly", which 
cannot lead to the therapeutic success sought. On the 
contrary, in this case there is even the risk that, on 
account of the "incorrect" movement sequence, there 
will be no improvement at all, or even a deterioration. 

Self -perception of a subject's own movements is 
often insufficient for adequate monitoring, since self- 
perception can be disturbed, for example on account of 
specific disturbances to the subject ! s health, it is, 
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furthermore, often not possible to visually observe the 
movements (rear view, side view) . Finally, when 
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carrying out a complex, dynamic procedure, self- 
perception may be overtaxed, in other words the patient 
cannot simultaneously concentrate both on 'correctly 
carrying out the complex movement sequence and on 
detecting any movement errors. In order to remedy this, 
it would be possible to enlist an external observer, 
that is to say a trainer or therapist, but this 
involves effort and is very expensive. Furthermore, it 
is possible to utilise mirrors and the like for 
continuous self -observation . The disadvantage in this 
case is that, in spite of everything, the actual ideal 
body position or the ideal movement sequence cannot be 
identified, in other words adequate monitoring cannot 
be achieved by this means either. Finally, there also 
remains the possibility of capturing the movement 
sequence by means of a video recording and subsequently 
observing and analyzing it. However, self -monitoring 
during the movement is not possible in this case 
either. 



diagnosis method in which the person who is training 
has to carry out a movement using a training device, a 
measurement recording being used to detect the movement 
and display it in the form of a curve representing the 
movement course on a monitor. With respect to this 
curve it is possible to insert a predetermined curve to 
be reconstructed by the person who is training. 



interactive movement training in which optimum movement 
sequences are stored in a memory. While the exercises 
are being carried out, a video camera captures an image 
of the person who is training, said image being 
superposed on the stored video sequences. The person 
who is exercising simultaneously sees himself and the 
optimum movement sequence on a monitor and can 
compensate for any deviations. What is disadvantageous 
here, however, is that the person who is training has 
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to adapt the speed at which he performs an exercise to 
the speed 
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at which the video sequence is reproduced. This 
reproduction speed is adjustable, however. 

US 3 408 750 describes an apparatus in which the 
position of a golf player is recorded by a video camera, 
a video recording of an optimum movement sequence 
simultaneously being displayed on a monitor. There is, 
however, no interactivity between the recorded movement 
and the real movement. A system for the insertion of an 
optimum trajectory in a game of basketball by a laser 
beam is disclosed in US 5 365 427. However, the targeted 
training and monitoring of individual movement sequences 
is not possible in this way. 

The invention is thus based on the problem of 
specifying a system of the type mentioned in the 
introduction which avoids the disadvantages mentioned. 

In order to solve this problem, a system of the 
type mentioned in the introduction is provided, 
according to the invention comprising a video camera 
and a monitor for outputting the recorded video image, 
and also a means for inserting at least one moving 
marker, indicating a predetermined movement or body 
position, into the video image, the insertion means 
being designed for detecting characteristic points, 
lines, contours or the like of the person who is 
performing a movement sequence and is shown in a 
recorded video image sequence, or of the person's area 
shown, and for automatically adapting the movement 
speed of the moving marker to the movement speed of the 
moving person, or of the person's area. 

On the one hand, the system according to the 
invention utilises the possibility whereby images that 
have been captured by means of the video camera can be 
reproduced 11 live" on the monitor, so that the person 
can follow the movement sequence directly on the 
screen. The 
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to insert into the live image supplied by the video 
camera one or more markers indicating the ideal body- 
position with regard to the movement sequence 
predetermined by the therapist for example. The patient 
is thus continuously shown the desired position with 
regard to the previously known movement sequence, which 
he can immediately compare with the current actual 
position in which he is in and which he can see from 
the live video image. The subject can thus identify 
deviations from the desired position indicated by means 
of the markers, and can immediately correct them. This 
enables the subject to identify and perform the 
"correct 11 movement, so that the therapeutic success to 
be attained by the movement training can actually be 
achieved. As the marker, it is possible to insert, by 
way of example , a point or alternatively, of course , a 
plurality of points, assigned, for example, to 
different body extremities, but also one or a plurality 
of lines, in particular in the form of a stylized 
person ("matchstick man"), or, alternatively, in the 
form of contour lines or the like. The user can also 
choose between these as desired, depending on which 
display form he personally prefers for self -monitoring . 
The movements to be carried out and the position of the 
markers are stipulated by the trainer or therapist 
according to e.g. medical standpoints. 



moving marker indicating a predetermined, ideal body 
movement. The marker moves in parallel with and at the 
same time as the body, in other words the subject is 
shown the ideal desired position at every instant, 
which he can compare with the actual position in 
accordance with his own video image. This is expedient 
when it is important not only to attain a specific body 
position, as in the case described above, but also for 
the body movement to follow an ideal movement line or 
direction . 



The insertion means is designed for inserting a 
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If the speed of the movement is not important 
in specific movement sequences, for example in the case 
of power training, according to the invention the 
insertion means designed for detecting characteristic 
points, lines, contours or the like of the person who 
is performing a movement sequence and is shown in a 
recorded video image sequence, or of the person's area 
shown, can be designed for automatically adapting the 
movement speed of a moving marker to the movement speed 
of the moving person or of the person's area. 

The means for inserting the marker can, 
according to the invention, be directly integrated in 
the video camera. In video cameras, the insertion of 
graphic elements, e.g. in the form of an overlay, into 
the video image is a known standard function with which 
e.g. the time or a date can be inserted as text into 
the video film. The means, a graphics processor, which 
is integrated, according to the invention, in the video 
camera merely has to be configured or programmed in 
accordance with the marker to be inserted in the case 
of the system according to the invention. As an 
alternative to this, it is also possible, of course, to 
integrate the means, that is to say the graphics 
processor, directly in the monitor or, alternatively, 
to use an interposed insertion means, for example in 
the form of a personal computer, which is arranged in 
the communications connection between the video camera 
and the monitor (e.g. a communications line). 

According to the invention, the insertion means 
can also be designed for inserting a marker which is 
stationary during the body movement. In other words, in 
the case of this invention alternative, during the body 
movement in which, by way of example, the right arm and 
the right leg are to be simultaneously swung into a 
specific position, the ideal end positions to be taken 
up respectively by the arm and leg are indicated. In 
this case, the subject recognizes whether he is now 
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actually swinging his arm or his leg to an extent such 
that he is attaining the therapeutically 
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ideal desired position, or whether his swing is too 
short or far, for example. 

As described, the marker or markers serves or 
serve for indicating an ideal desired body position. In 
other words, the position or size and the like of the 
marker must be adapted and related to the position and 
the size, etc. of the person shown in the video image. 
The "position and size' 1 of the person shown in the 
image depends, on the one hand, on the size of the 
person himself /herself and, on the other hand, on the 
setting of the video camera or the distance thereof 
from the person. Moreover on whether, by way of 
example, only a specific body area is to be displayed, 
for example only a leg which is to be moved in a 
targeted manner, and which is then moved into the video 
image using a zoom device of the video camera. 

To provide a simple possibility ensuring that 
the person is correctly positioned with respect to the 
video camera, in order that, with respect to the person 
shown in the video image, the markers are inserted at 
the correct location based on the size of the person 
shown in the image, according to the invention it is 
possible to insert one or more markers which serve as 
adjustment markers and, by way of example, specify 
where the top of the head and where the feet and the 
like must be positioned in the video image. The person 
who is training then merely has to choose his position 
with respect to the video cameras such that his head 
and feet and the like are congruent with the adjustment 
markers inserted into the video image. In addition to 
these markers serving for ad j ustment , the further 
markers indicating the movement or body position to be 
attained are then inserted. In this case, the person 
who is training must maintain a fixed position with 
respect to the video camera . 

In order to enable simple adaptation and 
correlation, according to the invention the insertion 
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means can be designed for detecting characteristic 
points, lines, contours or the like of the 



13 

M 

§1 

m 
y 
%j 

pi 

13 

§15 

hi 

o 



AMENDED SHEET 



GR 98 P 03666P 

PCT/DE 99/02567 - 7 - 

non-moving person shown in the recorded video image, or 
of the person 1 s area shown, and for automatically 
adapting the marker, in particular the latter' s size 
and/or insertion position, in a manner dependent on the 
detection result. The insertion means is thus able to 
use the video image to detect the relevant information 
with regard to the person shown or the person's area, 
so that, using appropriate processing technology, the 
marker, that is to say, for example, the size of the 
"matchstick man", can then be related to the size of 
the detected person. This is expediently done when the 
person is not moving, since it is then a simple matter 
to detect said person's characteristic points. 

As an alternative to this, it is possible for 
the insertion means to be designed for detecting 
characteristic points, lines, contours or the like of 
the person who is performing a movement sequence and is 
shown in a recorded video image sequence, or of the 
person's area shown, and for automatically adapting the 
marker, in particular the latter 's size and/or 
insertion position, in a manner dependent on the 
detection result. In this configuration of the 
invention, therefore, firstly a complete movement 
sequence is recorded by means of the video camera. This 
can be done under supervision, for example, so that the 
subject performs the movement in the best possible way. 
In this case, it is then possible at the same time to 
recognize what the subject is currently able to do, so 
that, if appropriate, in addition to the automatic 
adaptation, manual intervention may also be made in the 
representation sequence of the marker, which may 
likewise be provided according to the invention. In 
this way, in the manner of a "teach-in", the ideal 
movement specification, that is to say the insertion 
data of the marker, can thus be generated in accordance 
with the actual ability of the subject to move, and be 
specifically geared to said subject. The trainer or 
therapist can thus generate the specific desired 
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movement sequence for the respective subject, defined 
by the marker (s) . 

If the speed of the movement is not important 
in specific movement sequences, for example in the case 
of power training, according to the invention the 
insertion means designed for detecting characteristic 
points, lines, contours or the like of the person who 
is performing a movement sequence and is shown in a 
recorded video image sequence, or of the person's area 
shown, can be designed for automatically adapting the 
movement speed of a moving marker to the movement speed 
of the moving person or of the person's area. 
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As described, in addition to automatic adaptation/ 
variation, manual variability of the size and/or of the 
insertion position and/or of the movement speed of the 
marker may also be provided. 

In a further configuration of the invention, 
the insertion means may be assigned a storage means in 
which, for a plurality of different predetermined body 
movement sequences, the respective insertion data of at 
least one marker are stored and can be selected by the 
user as desired. This enables a subject who, in the 
context of his rehabilitation or training, has to carry 
out a plurality of different movement sequences to 
select the marker sequence intended for the respective 
movement sequence, so that said marker sequence is 
displayed. 

Further advantages, features and details of the 
invention emerge from the exemplary embodiment 
described below and from the drawings, in which: 

Figure 1 shows a system of a first embodiment, 

Figure 2 shows a system of a second embodiment, and 

Figure 3 shows a system of a third embodiment . 

The system according to the invention which is 
shown therein comprises a video camera 1, which is used 
to record the movements of a person 2 . The video camera 
1 is connected via a corresponding data line to a 
monitor 3 , on which the recorded video image 4 can be 
output live. As an alternative to the data line, line- 
free communication is also conceivable. An insertion 
means 5 is connected between video camera 1 and monitor 
3. The insertion means 5 serves for inserting into the 
video image 4 shown at least one marker indicating an 
ideal desired body position which 
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should ideally be taken up by the person 2 who is 
carrying out a specific movement sequence. In the 
example shown, a plurality of markers 6 in the form of 
points are inserted into the video image 4 . These 
points 6 can be perceived visually by the person 2. In 
the example shown, the markers 6 are assigned to the 
various body extremities. Two markers 6 are assigned to 
the feet, two further markers are assigned to the knees 
and the last two markers are assigned to the hands. 
From the coincidence or non-coincidence of the markers 
6 with the respective body parts of the person 2 in the 
video image 4, the person 2 can recognize whether or 
not his/her body position corresponds to the desired 
position predetermined by the markers 6. In the 
exemplary embodiment shown, the movement is performed 
correctly insofar as the position and the posture of 
the left arm correspond to the movement specifications. 
However, the posture of the right arm 7 deviates from 
the desired position since the arm 7' shown in the 
video image 4 is not congruent with the assigned marker 
6'. The person. 2 can immediately recognize this 
deviation from the desired position during the movement 
and then correct it accordingly, so that the subsequent 
movement sequence can be carried out in a manner 
approximated even further to the desired position. 

Figures 2 and 3 show two system variants in 
which the insertion means 5 is integrated in the video 
camera (figure 2), or alternatively in the monitor 
(figure 3) . In each case the means comprises an 
appropriately designed graphics processor which can be 
appropriately programmed or insertion of the markers. 
Furthermore, figure 2 shows stationary markers 6" which 
are inserted into the video image and serve for 
adjustment or positioning of the person with respect to 
the video camera 1. The person changes his/her position 
with respect to the video camera 1 until e.g. the head 
and feet of the person in the video image are congruent 
with the respective markers 6" . 
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The insertion means 5, as is provided e.g. in 
the systems according to figures 1 and 3, may 
furthermore be able 
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to detect, within the video image 4, characteristic 
points, lines or contours of the person shown. From 
this it is possible to identify the size, position, 
etc. of the person shown in the video image 4, and to 
correspondingly adapt the insertion of the markers 6, 
since the latter have to be related to the size of the 
person shown. It the person shown in the video image 
were, for example, represented only half as large, for 
example if the video camera 1 were arranged at a 
corresponding distance from the person, then if there 
were no change to the insertion positions of the 
markers 6 shown in the example, said markers would be 
inserted completely incorrectly, in other words an 
actual/desired position comparison would not be 
possible in this case. This adaptation can be effected 
automatically, this expediently being done when the 
person is not moving. 

In addition, instead of (or, if appropriate, in 
addition to) the automatic adaptation of the marker 
position and/or size, it is possible (as described with 
respect to figure 2) that the insertion means 5 can 
insert in the video image stationary markers serving 
for positioning e.g. the head and feet of the person 
and for adjustment. In that case, the person only has 
to position himself /herself relative to the video 
camera in such a way that the head shown in the video 
image and the feet are congruent with the respective 
markers. In this case, the person must maintain this 
taken-up position during the exercise. 

Furthermore,, the insertion means is designed 
for inserting stationary markers, which only define 
ideal end positions of the body, and for inserting 
markers which move with the person. If it is not 
important to adhere to a specific movement speed during 
the movement sequence that is carried out, the 
insertion means 5 is furthermore able to adapt the 
movement speed of the markers 6 in accordance with the 
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movement speed of the person. In the case of automati 
adaptation this adaptation is effected 
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when the person is moving. In addition, the respective 
parameters of the marker can also be varied manually in 
order to be able, as desired, to effect manual 
correction or adaptation. In addition to the embodiment 
5 of the markers 6 in the form of points which is shown 
in the example, they can, for example, also be inserted 
in the form of lines, e.g. in the form of a stylized 
person ("matchstick man") or the like. Finally, the 
insertion means 5 may also be assigned an, expediently^, 
10 integrated storage means in which the insertion data of 
the markers for different movement sequences to be 
carried out by the subject are stored, which can be 
p selected as desired by the subject. 

to 

El 

P 

ill 

a 

V 3 

s i a 



AMENDED SHEET 



